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DETAILED ACTION 
Continued Examination Under 37 CFR LI 14 
1 . A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1.17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) 
has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 
37 CFR 1.114. Applicant's submission filed on 05/10/07 has been entered. 

This communication is responsive to Amendment, filed 05/10/07. 

Claims 1-2, 7-23, 28-33, 37-38, 40-41 are pending in this application. In the 
Amendment, claims 1, 22, 37-38, 40-41 have been amended, claims 3-6, 24-27, 39, 42 have been 
cancelled. This action is made non-Final. 



Claim Rejections - 35 USC § 103 
2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1 .56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 
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3. Claims 1, 2, 7-15, 17-19, 21-23, 28-33, 37, 38, 40, 41 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Pan et al. (US Patent No. 6,993,246), in view of Jones et al. (US 
Patent No. 6,453,355). 

As to claims 1, 37, 40, Pan teaches a method of storing data, said method comprising the 
steps of: 

storing data, as one or more data samples (i.e. event el, ... event e6; event fl .... Event f5; 
See Fig. 2), in a media file (i.e. data stream #1, data stream #2, Fig. 2; that event was stored in 
the data file, col. 2 } lines 58-64) configured for use by a media player application in playing the 
data samples (i.e. video playback system ... a standard media player, col. 11, lines 45-52), (i.e. 
Another aspect of the present invention involves correlating more than two data streams and 
storing related time-stamped event data pointers to a storage device for later retrieval As 
previously mentioned, the time-stamps indicate a reference time when a particular event was 
detected while a corresponding data pointer indicates a location where data associated with that 
event was stored in the data file, col. 2, lines 58-64); 

storing, in an index file (i.e. a data set is maintained in an index file, col. 2, lines 11-26) 
associated with the media file, at least an offset value (i.e. positional information, col. 1, line 59 
to col. 2, line 5) for each of the data samples representing a location (i.e. positional information 
indicating where data associated with the events are located in a corresponding storage medium 
such as a digital file or where the data is located within the data stream, col. 1, line 59 to col 2, 
line 5) of each of the data samples in the media file wherein the media file further comprises 
additional information (i.e. data pointer, time, See Fig. 2) interspersed throughout the media file 
(i.e. correlating data among multiple data streams based on a use of time-stamps and related 
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positional information, col 1, lines 46-58), wherein the additional information comprises at least 
a timestamp (i.e. time; See Fig. 2) for each of the data samples (i.e. event el, ... event e6; event 
fl .... event f5; See Fig. 2), each of the timestamps (i.e. tl, t3, tlO t t6, t8, til, t2, t4, t5, t7, t9; See 
Fig. 2) indicating a capture time of an associated data samples (i.e. During a presentation of 
information at slide display device 580, an audio-video recording unit 585 captures details of 
for example, a corresponding slide presentation including a lecturer delivering a speech. A data 
stream generated by the audio-video recording unit 585 is captured for storage in a data file at 
data processing unit 560, col. 10, lines 39-51). 
Pan does not explicitly teach: 

wherein the additional information of the media file is used in reconstructing the index 
file upon corruption of the index file, the reconstructive index file comprising the offset values 
representing the location of each of the data samples in the media file. 

Jones teaches: 

wherein the additional information (i.e. hint track; Each hint track sample, such as hint 
track sample 40 5 -which describes how to form an RTP packet-may contain a header, and may 
reference some data from an associated media track, col 10, lines 32-46) of the media file is 
used in reconstructing the index file (i.e. index of the data index associated with the samples that 
use this sample description, col 22, lines 60-63; Therefore each media track in the movie should 
have an associated RTP hint track. In one embodiment of the present invention, each hint track 
contains a track reference back to the media track which it is streaming, col 24, lines 18-25) 
upon corruption of the index file (i.e. the original media data may be removed, col. 12, lines 50- 
60; error correction, col 25, lines 34-43), the reconstructive (i.e. reassemble media file, col 8, 
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lines 58-65) index file comprising the offset values (i.e. media sample offsets, See Table 5, col. 
30) representing the location of each of the data samples in the media file (i.e. The immediate 
mode permits the insertion of payload-specific headers (e.g. the RTP H.261 header). For hint 
tracks where the media is sent In the clear \ the mediasample entry may specify the bytes to copy 
from the media track, by giving the sample number, data offset, and length to copy. For 
relatively complex cases (e.g. encryption or forward error correction), the transformed data may 
be placed into the hint samples, and then hintsample mode may be used, which would be 
provided from the extradata field in the RTP sample itself, col 25, lines 34-43). 

It would have been obvious to one of ordinary skills of the art having the teachings of 
Pan and Jones at the time the invention was made to modify the system of Pan to include the 
additional information of the media file is used in reconstructing the index file upon corruption 
of the index file, the reconstructive index file comprising the offset values representing the 
location of each of the data samples in the media file, as taught by Jones. 

One of ordinary skills in the art would be motivated to make this combination in order to 
reassemble media file in view of Jones, as doing so, it would give the added benefit of having 
hint tracks permitted the development of new formats for new protocols without causing 
compatibility issues for existing servers or local playback. In addition, new media tracks may be 
added over the life of the file format while maintaining backwards compatibility (col. 11, lines 
63-67) as taught by Jones. 
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As to claims 22, 38, 41, Pan teaches a method of storing video (i.e. video frames, col 11, 
lines 45-52) and associated text data (i.e. textual data stream, col 11, lines 16-34), said method 
comprising the steps of: 

storing the video associated text data, as one or more data samples (i.e. event el, ... event 
e6; event fl .... Event f5; See Fig. 2), in a media file in accordance with first file format (i.e. 
closed captions for video or audio content, col. 11, lines 15-23; Another aspect of the present 
invention involves correlating more than two data streams and storing related time-stamped 
event data pointers to a storage device for later retrieval. As previously mentioned, the time- 
stamps indicate a reference time when a particular event was detected while a corresponding 
data pointer indicates a location where data associated with that event was stored in the data 
file, col. 2, lines 58-64), the media being configured for use by a media file player application in 
playing the video data (i.e. video playback system, col. 11, lines 45-52); 

storing, in a index file (i.e. a data set is maintained in an index file, col. 2, lines 11-26) in 
accordance with a second file format (i.e. audio samples, col. 11, lines 45-52), at least an offset 
value for each of the data samples (i.e. event el, ... event e6; event fl .... Event f5; See Fig. 2) 
representing a location of each of the one or more data samples in the media file (i.e. Another 
aspect of the present invention involves correlating more than two data streams and storing 
related time-stamped event data pointers to a storage device for later retrieval. As previously 
mentioned, the time-stamps indicate a reference time when a particular event was detected while 
a corresponding data pointer indicates a location where data associated with that event was 
stored in the data file, col. 2, lines 58-64); and 
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adding additional information (i.e. data pointer, time, See Fig. 2) interspersed throughout 
the media file (i.e. correlating data among multiple data streams based on a use of time-stamps 
and related positional information, col 1, lines 46-58), the media file including the additional 
information being readable by the media player application corresponding at least to first file 
format (i.e. video playback system ... a standard media player, col 11, lines 45-52), the 
additional information comprising at least a timestamp (i.e. tl, t3, tlO, t6, t8, til, t2, t4, t5, t7, t9; 
See Fig. 2) for one or more of the data samples (le; event el, ... event e6; event fl .... event f5; 
See Fig. 2), each of the timestamps indicating a capture time (i.e. During a presentation of 
information at slide display device 580, an audio-video recording unit 585 captures details of 
for example, a corresponding slide presentation including a lecturer delivering a speech. A data 
stream generated by the audio-video recording unit 585 is captured for storage in a data file at 
data processing unit 560, col 10, lines 39-51) of an associated data sample (i.e. The principles of 
the present invention have applications beyond note taking, text entry, or a close captioning 
system. One example is a video surveillance system for an automobile parking lot. This example 
system uses multiple video cameras. One set of cameras record the automobiles and associated 
license plates at the entrance and exits. Another set of cameras record the parking area. In 
current video surveillance systems, cameras continuously record video on tape. This results in a 
waste of video recording media. A more efficient system would record, for example, only for a 
few seconds after a motion detector indicates when a car has entered or left the garage or 
parking space. This type of sporadic recording renders it difficult to determine where an event is 
recorded on a videotape. Even a time-stamped videotape must be searched for the appropriate 
time match, col 13, line 62 to col 14, line 9). 



Application/Control Number: 10/748,334 Page 8 

Art Unit: 2167 

Pan does not specifically teach: 

wherein the additional information to the media file is used in reconstructing the index 
file upon corruption of the index file, the reconstructed index file comprising the offset values 
representing the locations of each of the data samples in the media file. 

Jones teaches: 

wherein the additional information (i.e. hint track; Each hint track sample, such as hint 
track sample 405-which describes how to form an RTP packet-may contain a header, and may 
reference some data from an associated media track, col 10, lines 32-46) of the media file is 
used in reconstructing the index file (i.e. index of the data index associated with the samples that 
use this sample description, col. 22, lines 60-63; Therefore each media track in the movie should 
have an associated RTP hint track. In one embodiment of the present invention, each hint track 
contains a track reference back to the media track which it is streaming, col 24, lines 18-25) 
upon corruption of the index file (i.e. the original media data may be removed, col. 12, lines 50- 
60; error correction, col 25, lines 34-43), the reconstructive (i.e. reassemble media file, col 8, 
lines 58-65) index file comprising the offset values (i.e. media sample offsets, See Table 5, col 
30) representing the location of each of the data samples in the media file (i.e. The immediate 
mode permits the insertion oj "pay load-specific headers (e.g. the RTP K261 header). For hint 
tracks where the media is sent 7/7 the clear \ the mediasample entry may specify the bytes to copy 
from the media track, by giving the sample number, data offset, and length to copy. For 
relatively complex cases (e.g. encryption or forward error correction), the transformed data may 
be placed into the hint samples, and then hintsample mode may be used, which would be 
provided from the extradata field in the RTP sample itself col 25, lines 34-43). 
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It would have been obvious to one of ordinary skills of the art having the teachings of 
Pan and Jones at the time the invention was made to modify the system of Pan to include 
wherein the additional information to the media file is used in reconstructing the index file upon 
corruption of the index file, the reconstructed index file comprising the offset values representing 
the locations of each of the data samples in the media file, as taught by Jones. 

One of ordinary skills in the art would be motivated to make this combination in order to 
reassemble media file in view of Jones, as doing so, it would give the added benefit of having 
hint tracks permitted the development of new formats for new protocols without causing 
compatibility issues for existing servers or local playback. In addition, new media tracks may be 
added over the life of the file format while maintaining backwards compatibility (col. 11, lines 
63-67) as taught by Jones. 

As to claims 2, 23, Jones teaches the additional information is used exclusively for 
reconstruction of the index file (i.e. reassemble media file, col. 8, lines 58-65). 

As to claims 7, 28, Pan teaches the additional information comprises a resolution of an 
associated sample (i.e. The consequences of latency depend upon the application and the desired 
time resolution at which queries are being made, col 12, lines 46-55). 

As to claims 8, 29, Pan teaches the information of the index file comprises frame rate 
variation information (i.e. video rate of 30 frames per second, col. 11, line 58 to col 12, line 2). 
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As to claims 9, 30, Pan teaches the additional information is stored as one or more 
dedicated samples of the media file (i.e. FIG 3 is a graph of sample trigger events as recorded 
over time according to sample data in the table of FIG 2. According to the principles of the 
present invention, events in one data file are indexed to events in a second data file based on 
time-stamps and corresponding data pointers, col. 7, lines 10-14). 

As to claims 10, 31, Jones teaches the media file is configured in accordance with the 
Microsoft AVI file format (i.e. Thus, according to one embodiment of the present invention, 
existing legacy formats such as "au" audio files, "AVI" audio/video files, col. 11, lines 14-29). 

As to claims 11, 32, Jones teaches the index file is configured in accordance with the 
Apple QuickTime file format (i.e. QuickTime files, col 11, lines 14-29). 

As per claim 12, Pan teaches the data is video data (i.e. video frames, col. 11, lines 45- 

52). 

As per claim 13, Pan teaches the data is text data (i.e. textual data stream, col 11, lines 

16-34). 

As per claim 14, Pan teaches the data is video data and associated text data (i.e. closed 
captions for video or audio content, col 11, lines 16-23). 
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As to claims 15, 33, Pan teaches the video and associated text data are captured for 
security purposes (i.e. The principles of the present invention have applications beyond note 
taking, text entry, or a close captioning system. One example is a video surveillance system for 
an automobile parking lot. This example system uses multiple video cameras. One set of cameras 
record the automobiles and associated license plates at the entrance and exits. Another set of 
cameras record the parking area. In current video surveillance systems, cameras continuously 
record video on tape. This results in a waste of video recording media. A more efficient system 
would record, for example, only for a few seconds after a motion detector indicates when a car 
has entered or left the garage or parking space. This type of sporadic recording renders it 
difficult to determine where an event is recorded on a videotape. Even a time-stamped videotape 
must be searched for the appropriate time match, col. 13, line 62 to col. 14, line 9). 

As per claim 17, Jones teaches a plurality of copies of a corresponding text string (i.e. 
text sample, col 27, lines 18-29) are included in each text sample of the media file (i.e. In the 
QuickTime file format, a track may be added to the movie by updating or copying and 
augmenting the meta-data. If the media data is in files separate from the meta-data, or optimized 
interleave is not required, this can be a relatively simple and efficient operation, col. 11, lines 
44-49). 

As per claim 18, Jones teaches a first copy of the text string (i.e. text sample, col. 27, 
lines 18-29) is configured in accordance with the AVI file format (i.e. Thus, according to one 
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embodiment of the present invention, existing legacy formats such as "au" audio files, "A VI" 
audio/video files, col 11, lines 14-29). 

As per claim 19, Jones teaches a second copy of the text string (i.e. text sample, col. 27, 
lines 18-29) is configured in accordance with the QuickTime file format (i.e. In the QuickTime 
file format, a track may be added to the movie by updating or copying and augmenting the meta- 
data. If the media data is in files separate from the meta-data, or optimized interleave is not 
required, this can be a relatively simple and efficient operation, col. 11, lines 44-49). 

As per claim 21, Jones teaches the index file contains a track referencing at least the 
media file (i.e. a hint track, col. 7, lines 47-64). 

4. Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over Pan et al. (US 
Patent No. 6,993,246), in view of Jones et al. (US Patent No. 6,453,355), and further in view of 
Obrador (US Patent No. 7,149,755). 

As per claim 16, Pan and Jones do not specifically teach each video sample is a separate 
JPEG file. 

Obrador teaches each video sample is a separate JPEG file (i.e. a motion JPEG 
compression format for audio/video works, col. 3, line 50 to col 4, line 14). 

It would have been obvious to one of ordinary skills of the art having the teachings of 
Pan, Jones, and Obrador at the time the invention was made to modify the system of Pan, and 
Jones to include each video sample is a separate JPEG file as taught by Obrador. 
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One of ordinary skills in the art would be motivated to make this combination in order to 
transmit the requested digital content to the user in a format that is suitable for rendering by a 
computer, a wireless device, or a voice device (col. 3, line 50 to col. 4, line 14) in view of 
Obrador, as doing so would give the added benefit of allowing a user to organize and browse 
through a selected collection of media objects by means of a set of links between media objects 
(col. 4, line 65 to col. 5, line 8) as taught by Obrador. 

4. Claim 20 is rejected under 35 U.S.C. 103(a) as being unpatentable over Pan et al. (US 
Patent No. 6,993,246), in view of Jones et al. (US Patent No. 6,453,355), and further in view of 
Boogaard (US Pub. No. 20030033325). 

As per claim 20, Pan and Jones do not specifically teach the step of inserting one or 
more empty samples into the media file to compensate for any missed samples. 

Boogaard teaches comprising the step of inserting one or more empty samples into the 
media file to compensate for any missed samples (i.e. The available empty frame(s) is or are 
available because the first next normal frame may need the space in the empty frame(s) to store 
data (bit reservoir), [0101 J). 

It would have been obvious to one of ordinary skill of the art having the teaching of Pan, 
Jones and Boogaard at the time the invention was made to modify the system of Pan- and Jones to 
include the step of inserting one or more empty samples into the media file to compensate for 
any missed samples as taught by Boogaard. 

One of ordinary skill in the art would be motivated to make this combination in order to 
avoid possible additional frames for a bit reservoir from being counted in the time coding of the 
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file in view of Boogaard ([0101]), as doing so would give the added benefit of pertaining more 
efficient distribution of unique audio and/or video files by combining the invention with the 
"client as server" principle, which uses (partial) recording of audio and/or video at the side of the 
individual user, while maintaining a method of control of the server as taught by Boogaard 
([0029]). 

Response to Arguments 

5. Applicant's arguments regarding the prior arts do not suggest a method for storing data as 
recited in amended independent claims 1, 22, 37, 38, 40, 41 have been considered but are moot 
in view of the new ground(s) of rejection. 



Conclusion 

6. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Miranda Le whose telephone number is (571) 272-41 12. The 
examiner can normally be reached on Monday through Friday from 8:30 AM to 5:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John R. Cottingham, can be reached on (571) 272-7079. The fax number to this Art 
Unit is 571-273-8300. 

Any inquiry of a general nature or relating to the status of this application should be 
directed to the Group receptionist whose telephone number is (703) 305-3900. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http :// pai r-direct . uspto . gov , Should you have questions on access to the Private 
PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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